1. Field of the Present Invention
The present invention generally relates to the field of integrated circuit functional verification and more particularly to a verification system and method that evaluate randomly generated tests based on the amount of functional coverage achieved relative to a given test specification prior to performing actual simulation.
2. History of Related Art
As the complexity of microprocessors and other complex integrated circuits has consistently increased over the years, the process of verifying each new design has accounted for an increasingly large percentage of the total resources required to design and manufacture a particular integrated circuit. Indeed, the verification of complex microprocessors with multiprocessing capability is now estimated to consume more time, labor, and other resources than the actual design of the integrated circuit.
Typically, functional verification is accomplished by generating a large number of test cases (test programs) developed manually by designers and verification engineers as well as with the help of various random and specific test generators and running these test programs on a simulator that attempts to mimic the operation of the microprocessor or other device. As the number of transistors, functions, registers, and other facilities in the integrated circuit have increased, conventional verification methods have responded by simply increasing the number of tests that are simulated. Unfortunately, generating a seemingly infinite number of tests is an inefficient and unreliable method of verifying the functionality of all components in the processor.
Typically, random test generators will generate a large number of test cases that are redundant from the perspective of the number of microprocessor states that are tested. In other words, random test generators tend to generate a large number of test cases that exercise the same or similar classes or closely related classes of functionality of the microprocessor despite cosmetic differences in the test cases. In the majority of verification activities, neither the custom (deterministic) test cases nor the randomly generated ones adequately address the interdependencies and interactions between system level components and functions.
In the early days of microprocessor development, inefficiencies in functional verification were tolerated because the size of the test space (measured, for example, by the number of states the microprocessor may assume) was sufficiently small. In addition, early microprocessors typically had fewer functional units than modem microprocessors, and the interactions between the few components and functions were well understood and controlled. The increasing number of functional units in microprocessors is significant from a verification perspective because interaction between functional units can no longer be ignored or only loosely verified by conventional verification methodologies.
The general purpose use of modem integrated circuits makes it impossible to predict and plan for the type of software applications that will run on them and thus the state and interdependence that will be exercised in the field are rather large and generally non-deterministic. Roughly speaking, the test space of a microprocessor is approximately equal to 2n where n represents the number of latches (state storage devices) within the microprocessor. From this approximation, it will be appreciated that the test space of microprocessors increases exponentially as the number of latches is increased.
The conventional approach to functional verification, in which increased complexity in a device is verified by simply increasing the number of tests that are simulated, is rapidly becoming unfeasible. The simulation process itself is resource intensive. In addition, because the input to a simulator in a conventional verification process is simply a large number of deterministic tests or randomly generated tests, the output of the simulation must be painstakingly evaluated to determine whether a particular simulation was successful in testing the intended functionality of the device.
It would, therefore, be highly desirable to implement a verification system in which the test specification produced by a test generator is evaluated in terms of the functional coverage value-added prior to simulation in an effort to reduce the number of tests that are actually simulated and to ensure that the tests that are simulated are likely to achieve additional verification of the device. Such a capability will result in more focused verification, shorter verification cycle and more efficient utilization of costly simulation cycles. In addition, it will provide a mechanism for more meaningful measurement and monitoring of the verification process, resulting in better scheduling and coordination with other design and manufacturing tasks.
As The problems identified are addressed by an integrated circuit functional verification method and system as disclosed herein. The method includes generating a test description comprising a set of test cases. The functional coverage achieved by the test description is then determined. The functional coverage achieved is then compared against previously achieved functional coverage and the test description is modified prior to simulation if the test description does not offer a delta coverage value add.
Rather than loosely coupled randomly generated test cases, this invention provides the architecture and method for coverage-directed pseudo random test generation for measurable targeted functional verification. In one embodiment, generating the test description comprises generating a test specification and providing the test specification to a test generator suitable for generating the test description. In one embodiment, the test description comprises a generic (i.e., generic format) test description and the generic test description is formatted according to a project test format specification if the coverage achieved by the test description satisfies the test specification.
In one embodiment, the functional coverage achieved by the test description is displayed in a graphical format. The test description is preferably added to a regression test database if the coverage achieved by the test description satisfies the test specification. Determining the functional coverage achieved by a test description may include estimating the coverage achieved based upon the test description, the test specification, a project specification comprising a high level model of the integrated circuit and coverage knowledge base.
The system according to the present invention preferably comprises a test generator, a coverage estimator, and a test coverage analyzer. The test generator is suitable for receiving a test specification and generating a test description responsive thereto. The coverage estimator is adapted to generate a test coverage metric indicative of the functional coverage achieved by the test description. The coverage estimator includes a coverage analyzer which performs high level simulation of the test description to generate a coverage signature.
In one embodiment, the system further includes a test specification optimizer configured to receive a test specification and adapted to produce an optimized test specification based on inputs from the coverage estimator and a project specification comprising a high level model of the integrated circuit. The optimized test specification is preferably adapted to produce a test description that achieves functional coverage of the integrated circuit that is equal to or greater than the functional coverage achieved by the original test description.
In one embodiment, the test generator generates a generic test description and the system includes a formatter adapted to modify the generic test description based on project specification information. In one embodiment, the coverage estimator utilizes a rule base indicative of a mapping between a test specification and a coverage model. The rule base is utilized to map the generic test description to a set of coverage states.